In recent years, digitization has been developed significantly also in the field of image capturing apparatuses with the development of the digitization of information. Especially, in image capturing apparatuses represented by digital cameras, as for an image capturing surface, a traditional film is replaced by a solid state image capturing device. A CCD (Charge Coupled Device) sensor, CMOS (Complementary Metal Oxide Semiconductor) sensor, or the like is used as the solid state image capturing device (hereinafter simply referred to as an “image capturing device”).
In this manner, the image capturing apparatus using the image capturing device takes in light from a subject through an optical system, extracts the light by converting it into electrical signals by the solid state image capturing device. Examples of such an image capturing apparatus include a video camera, a barcode reader, a mobile phone, a portable information terminal (PDA: Personal Digital Assistant), and an industrial camera, in addition to a digital camera.
The above-mentioned image capturing apparatus includes what is called an extended depth of field camera (EDoF: Extended Depth of Field) that extends the depth of field with the optical system. The depth of field indicates the range of distance in an optical axis direction of a lens in which it is acceptable to regard a subject at the distance away from the lens of the camera as being in focus.
A specific description is given of the extension of the depth of field by the optical system. A lens and a phase plate, which are included in the optical system, function to give aberration and add the point spread function (PSF: Point Spread Function) to the light of a subject incident on an image capturing device. For example, the lens gives spherical aberration, as the aberration, to the light of the subject incident on the image capturing device. The optical system causes an image captured by the image capturing device to be blurred by the aberration but makes the blur constant in a wide depth of field. Therefore, the image blurred by the optical system needs to be corrected in such a manner as to obtain a predetermined value of MTF (Modulation Transfer Function). MTF indicates a digitized value depicting the degree of fidelity to reproduce the contrast of the subject, in other words, the reproducibility of the contrast. In this case, the inverse transformation process of the point spread function is performed on the image blurred by the optical system to restore the image to a high resolution image. the inverse transformation process is implemented by correcting the blur of the image by performing a filtering process with an inverse transform filter on information of each pixel forming the image blurred by the optical system (hereinafter simply referred to as the pixel).
Moreover, an image captured by the image capturing device generally includes noise. Accordingly, a noise reduction process is also required. As an apparatus that reduces noise of such a captured image and corrects blur for restoration, proposed is an apparatus including a noise processing unit that generates a noise processed image by performing a noise process of reducing noise on an input image, and a restoration processing unit that performs a restoration process of reducing blur on the noise processed image (Patent Literature 1). The noise processing unit is configured to perform the noise process based on the characteristics of the restoration process.
However, the apparatus described in Patent Literature 1 has a problem in that the apparatus prevents deterioration of an image due to that the influence of the noise processing unit is amplified, and cannot fundamentally prevent amplification of noise. Moreover, resolution and the amount of noise are in the trade-off relationship. Therefore, there is also a problem that if the resolution of a lens is largely compensated, the amount of noise increases.
In view of the above, there is a need to provide an image capturing apparatus, image capturing system, and image capturing method that restore blur which is optical aberration while suppressing noise.